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The Problem of the Effeotlvo Definition of "random noguonco 1 * 

Mathematicians have always had difficulty in coming to agree- 
ment over what is meant by "randomness”. In order to agree on a 
formal model for a "random process", we have to agree on what intui¬ 
tive aspects of the matter we want to build into our system. The 

« 

most prominent point of agreement is that the process should be un¬ 
predictable, but this is in itself a very small beginning. The solu¬ 
tion that has become conventional in modern mathematics is based on 
the notion of "random Variable", a highly technical notion in which 

the basic prooess is represented as a certain kind of infinite func- 

# 

tion-space. This space contains all possible observed behavior 
sequences together with a "measure" structure which enables one to 
calculate the relative frequency of certain ("measurable") complex 
events. "Event" here usually refers to a whole class of behaviors. 

In this generally accepted theory, the individual "random 
sequence" plays a very small role, because its individual measure 
or probability is zero, in non-trivial cases. Thus, if one flips 
a fair coin, the probability of any particular sequence 

is evidently zero. Furthermore, for this 
situation all individual sequences are equally likely. Yet the par- 
fcioular sequence ' is "obviously" non-randoml 

If we can say that a particular sequence is not random, shouldn’t 
we be able to say that some other sequence is random? Until one 
becomes ever so sophisticated, this feeling lingers. But how could 
one characterize a single random sequenoe. One would have to be 


•3 





careful, for if the description is too specific, the sequence might 
lose the precious quality of controlled unpredictability. 

It would be very pleasing and useful, if one could set up a 
satisfactory theory in which it was meaningful to talk of an indi¬ 
vidual random sequence. It would be even more pleasing and useful 
on® could define this in some effective manner. 

It seems safe to say that we could not admit any computable 
sequences to the category of random sequences• For a computable 
aequenoe has the character that one needs only a finite amount of 
Information In order to be able to make perfect predictions 

On the other hand, it might seem safe to say that a sequence 
should be considered random If there is no way to make a "better 
——chance* prediction a bout its next term, given the past behavior 
The constraint that there be "no way to make a 'better than 
chance' prediction" has the right spirit, but It is far too vague 
to be useful as a mathematical condition. 

We will formalize the notion is a certain "effective" way and 
then show that In at least that sense, there will exist computable 
sequences which sa tisfy our definition of randomness . This result 
be viewed In two ways. On the one hand It shows the futility 
of trying to make effective such a definition, if one's goal is to 
obtain an absolute notion of randomness. On the other hand. It does 
show that there are computable sequences so Irregular that one can 

i. 

be constructed to defeat any such systematic attempt: that Is, there 

are arbitrarily "disordered" yet effeotively-defined "pseudo-random" 
sequences. 

« Indeed, the reader unfamiliar with the definition of computability 
can take this as the definition. 

* The name of R. Von MIses is associated with such attempts. See 
ref. [4]. 
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We will use a very crude measure of prediction ability. Let 
M be a (Turing) maohine which computes some binary function of 
finite binary sequences. That is, given any finite string of O’3 
and l’s, M will produce a 0 or a 1. Given some infinite sequence 
S(l), S(2), ... we can have M produce, for each term S(t), a 
digit M(t+1) which we will interpret as a"guess" about what will 
be the next term S(t+1) of the sequence S. We keep a "cumulative 
score 1 * G(t) for M , adding 1 to it for each correct guess and 
subtracting 1 for each inoorrect guess. Ifi M can consistently do 
" better than ohance" then we might find, in the long run, that 

lim *■ a > 0 

t-*» v 

* 

(assuming that the limit exists at all).) 

Clearly, for any particular maohine M we could find computable 
sequences S(t) which would i) force the limitto 0, or ii) force 
the limit to 1 unity, and iii) prevent the relative score from 
converging to any limit at all. On the other hand, given first any 
computable sequence we could find a machine M which perfectly pre¬ 
dicts the sequence (by using the same machine that generates the 
sequence)• The problem wV\at, will concern us is that of the case in 
which a countable collection of machines M. are all simultaneously 
trying to predict the sequence. Can we find a sequence which will 
be able to "baffle" all of them simultaneously? The answer is that 
if the countable class *M, is effectively defined then there exists 

an effectively defined sequence which will Baffle them all. 

•» Indeed, we would obtain a satisfactory notion of randomness if we 
accept a sequence as random if and only if it baffles all effective 
prediction methods. But since the latter cannot be effectively 
enumerated, we rejeot this as an unrealistic definition. 




Let us accept for the moment the definition that if a ■ 0 
for a particular machine and sequence, that machine has boon fooled 
by that sequenoe • Now suppose that a class of Turing machines 

has been defined effectively , that is, there is given a ’’master” 
machine M which, given the "integer •j f and a sequence S(0),S(1),.. 
S(t), computes what M. would give for that same sequence. Suppose 
further (but informally, now) that this collection has been so chosen 
ad to inolude all known tests for statistical regularity, so that 
there is at least some basis for thinking of the machines as actually 
trying to make predictions• 

It was shown by Minsky and Silver (Ref. 1) that under these 

f 

conditions there always exists a computable sequence for which 

I ; 

a *f ,0/ for every one of the maohines M.* The question of whether 

i . | 

this 1 remains true for machines which don’t have to guess every time 

t 

was 1 unanswered in [1]. It is felt that this case is important, be¬ 
cause it allows more flexible "statistician machines" which can wait 

for oertain special events and only then guess. Thus, a machine 
might wait until there was a run of 1,000,000 consecutive 0's, add 
then guess that the next term will be 0. This would be a sound 
policy in the real world, for if the process is really random, one 
will do no worse than ohanoe, while if the long run is due to a 
systematic defect in the experiment, that defeot is liable to remain 

operative for some further time. The theorem was extended to this 

( 

more general case by Levin,' and the proof below is essentially that 
in his thesis [2]• 

This oonoept of guessing is now extended to allow the guessing 
machines to guess only some of the time. This seems to be a fairer 




This means that we cannot use the simple score function described 
earlier, but must take the score relative to the number of actual 


guesses. 

We now state a number of definitions. The sequence to be pre¬ 
dicted is S(t) = S(l), S(2), ... The sequence--or rather, its 

"t 

initial segments--are examined by a sequence of Turing machines M,. 

It is required that each of these machines compute a total function 

over the set of all finite sequences. That is, given a finite sequence 

(of 0‘s and I's) the machine will eventually halt, giving an output. 

The n th guess of the machine M, is written Mj(n) and is the value M^ 

gives in response to the sequence [s(l$, ...,S(n-l)]. This value 

must be either 0, 1, or <E> (which signifies "no guess"). 

We keep a running score G.(t) of the number of correct 

J 

guesses minus the number of incorrect guesses. Let H.(t) be the number 
of times the machine has guessed rather than indicated <E>. Then we will 



say that if either 


lim £jil> . 0 


or if lim H.(t) < oo, 
t —*■ c* 3 ^ 


the machine M,(t) has failed to predict the sequence S(t). In the 
first case the machine guessed correctly no more than half the time. 


In the second case the machine stopped guessing and we cannot give it 


credit for being lucky on a finite number of guesses. In all other 

* 

cases we consider the machine to have made a significant prediction 


of S(t). 

Returning to the idea of random nets, one might hope to define 
a sequenoe as random if it can pass a certain family of tests. By 

f 

using the Turing Machines M., we restrict the tests to be effectively 
.computable. We have required that the test values themselves be ,. 

i 

defined—that the machines terminate each computation. ; 
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One attempt at such a definition would 3irnply include all 
possible guessing machines in the test family..A sequence whixh 
passes all such tests will surely appear random. Clearly these could 
exist no such co mputable sequence, but as shown by Church[3]*there 
must exist (non-computable) such sequences. However, thi 3 definition 

cannot be made effective, because the set of tests itself-the set 

of Turing machines which compute functions defined over the whole 

input range-cannot be effectively defined. Instead, we consider 

an arbitrary effectively enumerated sequence of tests-the sequence 

of machines defined above. Our theorem is 

Given any effectively enumerated sequence of test machines M., 

1 * J 

there exists a computable sequence S(t) which baffles all the M>. 

Since the sequence^M Aof machines is effectively enumerable, 
there exists a single machine that computes the same function as M. 

' J 

when supplied with the Integer J. That is, there is a machine U such 

. ; 

that Mj(t) - U(j,t). 

To show how the sequence S(t) is constructed, we begin with the 

1 ' ■ 

special case in which there is only a finite number, n, on machines. 
Assume that S has been computed for all t< tQ. Thon Y/e can compute 
f° r ail 1^ n and 1£ t* tQ. For each t the guesses of the 
n machines form an n-vector V[t] « [U(l,t),U(2, t),...,u(n,t)]. There 

are 3 n possible values for V[t]. To compute S(t Q ) we count the. number 

' • #. 

of values of t for which V[t]=V[t 0 ] and t<t Q . Assign the value 0 or 1 
to S(t q) as this number is even or odd. 

To illustrate, suppose there are only two machines, and they 
have just made their 10 tiv guess: 

*But not for the optional guessing case. If we remove the con¬ 
dition in the theorem below, that the machines be effectively 
enumerable, we .obtain Church's result, since the construction 
carries through (npn-effectively) without this condition. 
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M x (t) 
M 2 (t) 
S( t) 


O' 


10 


The vector V [10] is (1,0)• .Since this identical vector has already 
ooourred exactly twice before (V[10j= V[7] * V[l]), and two is even, 
the value of S(10) should be 0* 

Now consider any vector type, for instance the vector type of 
7[1]» VC73 * find V[10]. The value of S(t) alternates strictly on 
the subsequenoe 1, fl, 10, • If the correct-incorrect score G,(t) 

v 

were kept only on this subsequence of t, then it could never become 
greater than 1* But there are at most 3* 1 such subsequences, corres- 
ponding to the 3 vector types when there are n maohines* Therefore: 

2) Gj(t) < $ n 

If for a given machine M., lim H,(t) ■« », this implies: 

J t-*» J 


3) 


lim 


G 4 ( t) 


Therefore M. has failed, to prediot S(t), 

To extend this technique to infinite sets of machines, we 
cannot consider infinite vector^, but we oan increase the number of 
machines under consideration so as to include each of them eventu¬ 


ally 


A value of S(t) obtained by applying the above recursive scheme 


to the first J machines will be called S*(j,t), The number J will 
be called the depth of the computation. In general, the depth tends 


r 
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to Increase with Increasing t. By deoiding on the correct depth 
for eaoh t, we can oompute S(t) by setting S(t) * S*(j,t). 

i 

The rule for computing the depth Is now given. Let t„ be fixed 
and consider the sequence S»(l,t ), S‘(2,t 0 ), ... • In computing 

a count was made on the number of earlier vectors that are 
identical to V[t Q ] * (S'(l,t Q ),...,S'(J,t Q )) • If this frequency 
count is greater than 3*^, then the computation of S *(J,t Q ) will be 
called excessive. The correct depth (J) Is now given as the smallest 
value of J for which S*( j,^) is not excessive. 

Having stated the algorithm, we must pow prove that it works 


by establishing the following facts: 

1. We shall consider a particular machine M n and prove that 
It cannot predict the sequence S(t). If M makes only finitely 
many guesses, then there is nothing to prove. We assume that It 
makes Infinitely many guesses for the rest of this proof. 


2. For each guess M (t), S(t) was made at a particular level 

* 

j, and for a particular vector type V[t] * (M^(t),...,Mj(t)). At 
level j there are 3j different vector types. At most 3^ computa¬ 
tions are made for each vector type before it becomes excessive. 
Therefore, at most 3^ computations are made at each level. 


This Is to say, that if we rewrite S(t) to indicate the level 


of eaoh computation by writing S'(j,,l), S*(J~,2)then each 
Integer j will occur at most 3^ times. 

3• There are an infinite number of values of t for which 


M (t) • Since only a finite number of them are made at any one 

l«wl and alnoa no laval can ba akippad, M n gueaaaa at all lavala. 

In faot, we can establish a minimum number of guesses atclevel j 


■* 


by the following reasoning: 


a. M guesses at least once at level J+i. This m^ano tb«t 


for some t, S»(J,t) ia .excessive' for a vector V[t] « C, (t), „ t ,, M . (t) 

x j 

such that M R (t) ■ 0 or 1, Let T(j) be the least such fc„ Th--n thf 
vector Vfc] has occurred 3 2 ^ times before time T(j). Note that the 
sequence forms a strictly monotonic unbounded sequence. 

b, After the vector V[t] occured times the level 

(j-i) became excessive. Therefore 3 2 ^ - = -g3 2 ^ guesses, 

at least, have been made at level j. 

4. We now wish to estimate 


H(t) 


Let T be the tim? such 


that for t>T, all guesses are made at level n or higher. (We are 

considering the score for machine M .) For all levels J<n the 

* 

machine may do very well. But even if all its guesses are correct, 
there are only finitely many of them. To be precise, these can 

t 

add at most T to the score G. £'5j. 

Starting with level n, M guesses right only half the time 
• • • 

on each vector type. At level j there are 3 ^ vector types and at 

8,23 

v- 

change by at most 3 ^ at level J, while H(t) must increase by at 

jMJ 

3- 


least ■^3“* J guesses are made. (See argument 3b.) Hence G(t) can 


least ^3 


5 . Now for each t find the j = J(t) such that T(J) * t < T(j+i) 


that is, the largest j 3uch that T(j) ^ t. We can then bound G(t) 

G(t) s t + 3*J 

> / 

3=n . 

for no guesses have been made at levels higher than J(t). We 
can set a lower bound on H(t) by the argument in (4) above: 

O *J (t )“ 1 A 1 
u/«-\ -V O / -3 2 J 


by 





f* 


Now, by ( 3 ), as t -+OO f j(t) 


0 s lim 
t 


a(t) 


H(t) 


£ lim 
J(t) 


oc> , so we have? 

J(t) 

T + £3 J 


CO 


8 J(t)-i 

9 B 

j 53 ! 


2 j 


Now if we replace all expressions of the form M.(t) by the 

J 

corresponding U(j,t), the construction becomes effective, S(t) i3 
a computable sequence, and the theorem is proved« 
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